Community Review — SP8192 + Banking + Triple Recurrence + Parallel Residuals + Muon 0.97 + TTT

Thanks @aryanbhosale — this is the same general stack family as your other submissions today (#1540 LoRA TTT + varlen) and the #1493 lineage. Clean compliance read below.

What I found (head SHA 9304879bb01246a0f8afa11e9f54e13f7e8f246b, records/track_10min_16mb/2026-04-11_SP8192_Banking_ParResid_TripleRecur_Muon97_TTT/train_gpt.py, decoded from the import lzma as L,base64 as B self-extracting shim — 58,957 bytes / 622 lines of actual source):

• eval_val_sliding — standard stride-64 sliding window under torch.inference_mode(), stock byte-accounting
• eval_val_ttt — score-first-per-chunk (chunk scored in no_grad pass, then SGD on scored chunk, is_last_chunk guard, cosine LR across chunks)
• Banking via parameter sharing across triple-loop passes (per PR title) — structural, no eval-time leakage concern
• Triple recurrence following the Record: ParallelResiduals + MiniDepthRecurrence, 1.1063 BPB / 1.8679 nats, -0.0072 vs PR #1179, -0.0143 vs merged SOTA #1204 (msisovic) + Record: Triple Loop + Fused Kernels + Parallel Residuals + N-gram Tilt; val_bpb 1.08309 (5-seed mean) #1420 (abaybektursun) pattern
• Parallel residuals following the Record: ParallelResiduals + MiniDepthRecurrence, 1.1063 BPB / 1.8679 nats, -0.0072 vs PR #1179, -0.0143 vs merged SOTA #1204 + GPT-J lineage
• Muon 0.97 momentum — optimizer tuning delta
• No prequant_ttt_adapt_adamw(val_tokens, ...) call, no scored-region SLOT, no full_key n-gram cache

CPU smoke test (CT2038 proteus-engine, 2026-04-11):

IMPORT_OK               seconds=6.00
HP_NUM_LAYERS           11
HP_MODEL_DIM            512
HP_VOCAB_SIZE           8192
HP_QK_GAIN_INIT         5.0
HP_CODE_BYTES           19760 (shim; decoded 58,957 B)
SMOKE_TEST_PASS

Compliance summary:

• TTT: legal (score-first-per-chunk, same shape as Record: SP8192 + Pre-Quant TTT — val_bpb 1.07948 (3-seed mean) #1416 / Record: SP8192 + Pre-Quant TTT + QK-Gain 5.0 + Depth Recurrence + MuonEq-R — val_bpb 1.0791 (3-seed mean) #1423 / Record: SP8192 + Improved Parallel Residuals + Muon 0.97 + LR 0.03 + Legal TTT — val_bpb 1.07785 (3-seed mean) #1541)
• SLOT: not present
• N-gram cache: not present
• Pre-Quant TTT on val: not present
• Byte accounting: stock SP8192 — clean
• Credit lineage: banking / triple recurrence / parallel residuals are architecturally iterative on Record: ParallelResiduals + MiniDepthRecurrence, 1.1063 BPB / 1.8679 nats, -0.0072 vs PR #1179, -0.0143 vs merged SOTA #1204 / Record: Triple Loop + Fused Kernels + Parallel Residuals + N-gram Tilt; val_bpb 1.08309 (5-seed mean) #1420 / Record: SP8192 + 3-Layer Recurrence + Parallel Residuals + QK-Gain 5.25 + Legal TTT — val_bpb 1.0810 (3-seed mean) #1493 — worth an explicit credit line in the README if it isn't already there

Verdict: LOOKS CLEAN.

Recommendation to @cocohearts @valerio-oai @0hq @yuzhougu-oai @notapplica: MERGE. Incremental improvement on the SP8192 + parallel-residuals + triple-recurrence lineage, clean eval path, no compliance flags. The architectural delta vs #1493 / #1533 / #1541 is small enough that the 3-seed std should be disclosed in the PR body for the statistical claim; on the static review nothing blocks landing.

Reviewed by @MatoTeziTanka — The Agora. CPU smoke test (CT2038 proteus-engine, 2026-04-11): IMPORT_OK 6.00s, SMOKE_TEST_PASS. Decoded source statically reviewed for the standard compliance axes — no flags. Full forward-pass / artifact gauntlet skipped (heavy architecture, CPU-bound past the budget). AI tooling: review drafted with Claude Code (Opus); batch-9 subagent quota exhausted so this review was authored in the main session. SHA 9304879bb01246a0f8afa11e9f54e13f7e8f246b.